System and method for medical procedure optimization

ABSTRACT

A method for optimizing execution of a medical procedure comprises obtaining medical procedure input associated with a medical practitioner identifier, a medical procedure identifier, and a medical procedure room identifier for execution of the medical procedure. A virtual image guidance for execution of the medical procedure based on the received medical procedure is provided. The method comprises acquiring data associated with usage of an item and an equipment during the medical procedure and analyzing prestored data associated with usage of the item and the equipment in one or more reference medical procedures corresponding to received medical procedure input. The method comprises comparing the acquired data associated with usage of at least one of the item and the equipment during the medical procedure with the analysed prestored data in one or more reference medical procedures and providing recommendation for optimizing the execution of the medical procedure based on the comparison.

BACKGROUND OF THE INVENTION

According to data from the National Center for Health Statistics, nearlyfifty (50) million surgical inpatient procedures were performed in 2009in the United States alone, and that number continues to grow. The setupand logistics of a medical procedure room and the procedure itselfshould be optimized for efficiency and procedure predictability.Additionally, although there may be a technique for a particularprocedure that is favored by the medical community, an individualmedical practitioner may have slight variations that he or she prefersand that make the practitioner's performance of that procedure moreefficient or predictable.

Before a medical procedure begins, the medical practitioner's preferredtechnique, as well as the medical procedure room setup, must becarefully planned. Additionally, proper medical procedure room setupalso depends on the size, shape, and configuration of the medicalprocedure room. Thus, there is not a “one size fits all” approach tomedical procedure room setup or performance of the medical procedure.Further, medical practitioners must be able to quickly and accuratelydetermine whether all required equipment has been placed in the medicalprocedure room, is in the right location, and that the providedequipment is of the correct type, size, and/or number to suit thepatient and ensure the medical procedure is performed safely andcorrectly. Not only must the fixed or semi-fixed equipment be properlyarranged prior to commencement of the procedure, but chargeable supplies(for example, sutures, sponges, clips, medical implants, screws, rods,arthroplasty devices, stimulators, needles, scalpel blades, catheters,drill bits) and disposable supplies (for example, gauze, gloves, liners,needles, syringes, and tubing) should be carefully tracked for billingand supply analysis and inventory management, as well as to make surethat the supplies are used in the correct order, in the correct amount,and in the correct way. During a medical procedure that demands the fullattention of the medical practitioner and other medical personnel, thetracking of such equipment can easily be ignored or incorrectly logged.Further, information about the number and type of supplies used duringthe procedure may be useful to the medical practitioner, medicalpersonnel, and/or medical facility for efficiency evaluation,practitioner success rates, cost per procedure, patient variation andresponse, procedure improvement, and/or other characteristics over time.

Finally, most medical procedures demand that the steps be performed in aspecific order, at a specific speed, with correctly sized implantsand/or tools, and the like. Regardless of how well trained the medicalpractitioner is, human error can occur, especially when performing newor complex procedures. Therefore, a medical practitioner must paycareful attention to the sequence of medical procedure steps, the numberand characteristics of all tools and supplies used, the medicalprocedure room setup, and patient needs, to name a few.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed invention and explainvarious principles and advantages of those embodiments.

FIG. 1 is a block diagram of an exemplary medical procedure system inaccordance with some embodiments.

FIG. 2 is a block diagram of an optimization system in accordance withsome embodiments.

FIG. 3 is a block diagram of a communication device for use within theoptimization system of FIG. 2 in accordance with some embodiments.

FIG. 4 is a block diagram of a mixed reality device for use within theoptimization system of FIG. 2 in accordance with some embodiments.

FIG. 5 is a block diagram of an optimization computing device for usewithin the optimization system of FIG. 2 in accordance with someembodiments.

FIG. 6 is a flow diagram of a method for use in medical procedureoptimization in accordance with some embodiments.

FIG. 7 is a flow diagram of a method of medical procedure optimizationin accordance with some embodiments.

FIG. 8 is a functional block diagram of a medical procedure optimizationsystem in accordance with some embodiments.

FIG. 9 is a flow diagram of a method of extended continuous procedureoptimization using the functionality of FIG. 8 in accordance with someembodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

In one aspect, a system for optimizing execution of a medical procedureis described. The system comprises a procedure module configured toobtain at least one medical procedure input associated with a medicalpractitioner identifier, a medical procedure identifier, and a medicalprocedure room identifier for execution of the medical procedure. Theprocedure module is further configured to provide virtual imageguidance, via one or more mixed reality devices, for execution of themedical procedure based on the received at least one medical procedureinput. The system further comprises a procedure input module configuredto acquire, via the one or more mixed reality devices, data associatedwith usage of at least one of an item, and an equipment during themedical procedure. The system further comprises a machine learningmodule configured to analyze prestored data associated with usage of atleast one of the item and the equipment in one or more reference medicalprocedures corresponding to the one or more of the medical practitioneridentifier, the medical procedure identifier, and the medical procedureroom identifier. The machine learning module is further configured tocompare the acquired data associated with usage of at least one of theitem and the equipment during the medical procedure with the analyzedprestored data associated with the respective at least one of the itemand the equipment in one or more reference medical procedures, andprovide recommendation for optimizing the execution of the medicalprocedure based on the comparison.

In another aspect, a method for optimizing execution of a medicalprocedure is described. The method comprises obtaining, by a proceduremodule, at least one medical procedure input associated with a medicalpractitioner identifier, a medical procedure identifier, and a medicalprocedure room identifier for execution of the medical procedure. Themethod further comprises providing, by the procedure module, virtualimage guidance, via one or more mixed reality devices, for execution ofthe medical procedure based on the received at least one medicalprocedure input. The method further comprises, acquiring, by a procedureinput module, via the one or more mixed reality devices, data associatedwith usage of at least one of an item and an equipment during themedical procedure. Further, the method comprises analyzing, by a machinelearning module, prestored data associated with usage of at least one ofthe items and the equipment in one or more reference medical procedurescorresponding to the one or more of the medical practitioner identifier,the medical procedure identifier, and the medical procedure roomidentifier. The method further comprises comparing, by the machinelearning module, the acquired data associated with usage of at least oneof the items and the equipment during the medical procedure with theanalyzed prestored data associated with the respective at least one ofthe item and the equipment in one or more reference medical procedures,and providing recommendation for optimizing the execution of the medicalprocedure based on the comparison.

Referring now to FIG. 1, a block diagram of an exemplary medicalprocedure system 100 is shown. In one embodiment, the medical proceduresystem 100 generally includes a medical procedure room 102 and anoptimization system 104. The configuration of the medical procedure room102 and/or optimization system 104, including physical placement ofcomponents, inventory and supplies, medical equipment used, medicalpersonnel 130 involved, networking of equipment, and othercharacteristics may be referred to as a medical procedure room setup andis indicated in FIG. 1 with reference number 106. The medical procedureroom 102 shown in FIG. 1 is a non-limiting example of a medicalprocedure system 100, and it will be understood that the systems andmethods disclosed herein are not limited to the number, type, placement,and configuration of items of equipment, components, medical personnel130, and/or other elements shown. Additionally, although shown withinthe medical procedure room 102 in FIG. 1, it will be understood that oneor more of the items of equipment, components, medical personnel 130,and/or other elements of the medical procedure room setup 106 may bephysically located outside or external to the walls of the room 102 andstill be considered to be part of the medical procedure room setup 106.

Continuing to refer to FIG. 1, in one non-limiting example, the medicalprocedure room 102 may include a medical procedure table 108, one ormore auxiliary tables or stands 110 (such as a Mayo stand), one or morestorage closets or rooms 112, nurse workstations 114, back tables 116,anesthesia systems 118, electrocautery systems 120, enabling technologysystems or workstations 122 (for example, but not limited to,microscopes, robotic surgical systems, networked robotics, illuminationsystems, or the like, and accompanying monitors or displays), user inputdevices 124 (such as mobile devices, tablets, or any communicationdevice now known or in the future developed), biometric readers 126,and/or wireless transceivers 128, as well as medical personnel 130 (forexample, but not limited to, surgical technicians, surgical team members(such as medical doctors and nurses), and anesthesiologists). In someembodiments, at least one member of medical personnel 130 wears a mixedreality device 132 during the medical procedure, and each mixed realitydevice 132 is in wireless and/or wired communication with theoptimization system 104. As noted above, it will be understood that themedical procedure system 100 and medical procedure room setup 106 is notlimited to those items shown in FIG. 1, and may include any number ofitems of equipment, electronic devices, imaging devices, surgical robotsor other systems, lights, or any other devices and/or supplies preferredor required by the medical practitioner and/or other medical personnel130.

Continuing to refer to FIG. 1, the exemplary medical procedure roomsetup 106 may be a preferred medical procedure room setup of thepractitioner performing the medical procedure, based on the type ofprocedure, number of medical procedure room personnel, available roomlayout and dimensions, and other factors. As discussed in greater detailbelow, the layout of the preferred medical procedure room setup 106 maybe uploaded or input into the optimization system 104 and may bedisplayed to a medical practitioner and/or medical personnel 130 throughone or more mixed reality devices 132 of the optimization system 104.Further, in one embodiment, inventory of disposable items, chargeableitems, and/or items that will remain in the patient is also uploaded orinput into the optimization system 104 for tracking, analysis, and/orinventory management. In one embodiment, the optimization system 104 maysuggest, or at least partially suggest, to the medical practitionerand/or medical personnel 130 an optimized medical procedure room setup106, amount and type of inventory and items of medical equipment, andother characteristics to enhance efficiency of the medical procedure.For example, such suggestions may be based at least in part on userinput and/or data collected by the optimization system 104 from previousmedical procedures (hereinafter interchangeably referred to as referencemedical procedures) of the same type or in the same medical procedureroom. Additionally, or alternatively, each practitioner's preferredmedical procedure room layout for each procedure may be stored in theoptimization system 104 as a default setup and suggested to a userand/or the practitioner when planning a new setup for the same orsimilar medical procedure. As the medical practitioner, medicalpersonnel 130, and/or other user of the optimization system 104 performseach step of the procedure and uses, moves, or removes each item ofequipment and/or inventory, such activity is logged by the optimizationsystem 104 for later analysis, inventory replenishment, education,procedural support, or other purposes.

FIG. 2 is a block diagram of an optimization system 104 in accordancewith some embodiments. Specifically, the optimization system of FIG. 2may be the optimization system 104 of FIG. 1. The optimization system104 provides for medical practitioner specific room organization, setup, supply, logistics, tracking and performance across surgical andmedical procedures with features for augmented reality, artificialintelligence and machine learning, as will be described further inaccordance with some embodiments hereinbelow. As shown, the optimizationsystem 104 includes one or more communication devices 124, one or moremixed reality devices 132, at least one optimization computing device206, a network 210, and one or more remote connections 208.

The optimization computing device 206 may be communicatively coupled to,and receive information from, the one or more communication devices 124,the one or more mixed reality devices 132 and the one or more remoteconnections 208. Communication between the optimization computing device206 and various components can occur through the network 210. In someembodiments, the network 210 is, for example, a wide area network (WAN)(for example, a transport control protocol/internet protocol (TCP/IP)based network), a cellular network, or a local area network (LAN)employing any of a variety of communications protocols as is well knownin the art.

Each of the one or more communication devices 124 operates as a userinterface for one or more medical procedure room personnel as will befurther described with respect to FIG. 3.

Each of the one or more mixed reality devices 132 further operates as auser interface for one or more medical procedure room personnel as willbe further described with respect to FIG. 4.

The one or more remote connections 208 interact with the optimizationcomputing device 206 via the network 210 to receive and provideinformation external to the medical procedure room 102. The one or moreremote connections 208 may be one or more distribution agentsincorporated within the optimization system 104 or independentlyconnected. The distribution agents may include, for example, but are notlimited to, one or more of buyers, purchasing groups, pharmacies,anesthetic components, sterile processing departments (SPD), cleaning,storage, management, and/or any equivalent general processingdepartment. The one or more remote connections 208 further may be one ormore collaborators which may be technology collaborators or specialistcollaborators and the like. The collaborators for example may include,but are not limited to, one or more of radiology, anesthesiology,fluoroscopy, electrophysiology, robotic and navigational systems, x-rayequipment techs, and the like. The one or more remote connections 208may be one or more educators including, for example, but not limited toresearchers, universities, training facilities, clinical trials, and thelike.

In operation, the optimization computing device 206 optimizes theorganization, preparation, and set up of a medical procedure space forefficient and predictable execution of one or more medical procedures.The optimization computing device 206, in some embodiments, operates tooptimize pre-preparation of a procedure, orientation of medicalpersonnel 130 for a procedure, and location and identification ofequipment for a procedure.

FIG. 3 is a block diagram of one exemplary embodiment of a communicationdevice 124 for use within the optimization system 104 of FIG. 2 inaccordance with some embodiments. The communication device 124 iselectrically and/or communicatively connected to a variety of otherdevices and databases as previously described with respect to FIG. 2herein. In some embodiments, the communication device 124 includes aplurality of electrical and electronic components, providing power,operational control, communication, and the like within thecommunication device 124. For example, in one embodiment, thecommunication device 124 includes, among other things, a communicationdevice transceiver 302, a communication device user interface 304, acommunication device network interface 306, a communication deviceprocessor 308, a communication device memory 310, and one or morecommunication device sensors 320.

It should be appreciated by those of ordinary skill in the art that FIG.3 depicts the communication device 124 in a simplified manner and apractical embodiment may include additional components and suitablyconfigured logic to support known or conventional operating featuresthat are not described in detail herein. It will further be appreciatedby those of ordinary skill in the art that the communication device 124may be a personal computer, desktop computer, tablet, smartphone,wearable device (wrist worn, eye worn, and the like), or any othercomputing device now known or in the future developed. It will furtherbe appreciated by those of ordinary skill in the art that thecommunication device 124 alternatively may function within a remoteserver, cloud computing device, or any other remote computing mechanismnow known or in the future developed.

The components of the communication device 124 (for example 302, 304,306, 308 and 310) are communicatively coupled via a communication devicelocal interface 318. The communication device local interface 318 maybe, for example but not limited to, one or more buses or other wired orwireless connections, as is known in the art. The communication devicelocal interface 318 may have additional elements, which are omitted forsimplicity, such as controllers, buffers (caches), drivers, repeaters,and receivers, among many others, to enable communications. Further, thecommunication device local interface 318 may include address, control,and/or data connections to enable appropriate communications among theaforementioned components.

The communication device processor 308 is a hardware device forexecuting software instructions. The communication device processor 308may be any custom made or commercially available processor, a centralprocessing unit (CPU), an auxiliary processor among several processorsassociated with the communication device processor 308, asemiconductor-based microprocessor, or generally any device forexecuting software instructions. When the communication device 124 is inoperation, the communication device processor 308 is configured toexecute software stored within the communication device memory 310, tocommunicate data to and from the communication device memory 310, and togenerally control operations of the communication device 124 pursuant tothe software instructions.

The communication device user interface 304 may be used to receive userinput from and/or for providing system output to the user or to one ormore devices or components. User input may be provided via, for example,a keyboard, touch pad, and/or a mouse. System output may be provided viaa display device, speakers, and/or a printer (not shown). Thecommunication device user interface 304 may further include, forexample, a serial port, a parallel port, an infrared (IR) interface, auniversal serial bus (USB) interface and/or any other interface hereinknown or in the future developed.

The communication device network interface 306 may be used to enable thecommunication device 124 to communicate on a network, such as thenetwork 210 of FIG. 2, a wireless access network (WAN), a radiofrequency (RF) network, and the like. The communication device networkinterface 306 may include, for example, an Ethernet card or adapter or awireless local area network (WLAN) card or adapter. Additionally, oralternatively the communication device network interface 306 may includea radio frequency interface for wide area communications such asLong-Term Evolution (LTE) networks, or any other network now known or inthe future developed. The communication device network interface 306 mayinclude address, control, and/or data connections to enable appropriatecommunications on the network.

The communication device memory 310 may include any non-transitorymemory elements comprising one or more of volatile memory elements (forexample, random access memory (RAM), nonvolatile memory elements (forexample, read only memory “ROM”), and combinations thereof. Moreover,the communication device memory 310 may incorporate electronic,magnetic, optical, and/or other types of storage media. Note that thecommunication device memory 310 may have a distributed architecture,where various components are situated remotely from one another, but canbe accessed by the communication device processor 308. The software inthe communication device memory 310 may include one or more softwareprograms, each of which includes an ordered listing of executableinstructions for implementing logical functions. The software in thecommunication device memory 310 includes a suitable communication deviceoperating system 314 and one or more communication device applications316. The communication device operating system 314 controls theexecution of other computer programs, such as the one or morecommunication device applications 316, and provides scheduling,input-output control, file and data management, memory management, andcommunication control and related services. The one or morecommunication device applications 316 may be configured to implement thevarious processes, algorithms, methods, techniques, and the likedescribed herein.

The communication device memory 310 further includes a communicationdevice data storage 312 used to store data. In the exemplary embodimentof FIG. 3, the communication device data storage 312 is located internalto the communication device memory 310 of the communication device 124.Additionally, or alternatively (not shown), the communication devicedata storage 312 may be located external to the communication device 124such as, for example, an external hard drive connected to thecommunication device user interface 304. In a further embodiment (notshown), the communication device data storage 312 may be locatedexternal and connected to the communication device 124 through a networkand accessed via the communication device network interface 306.

In operation, information for storage in the communication device datastorage 312 may be entered via the communication device user interface304. Alternatively, information for storage in the communication devicedata storage 312 may be received from the optimization computing device206, the mixed reality devices 132, or the remote connections 208 viathe communication device transceiver 302. Alternatively, information forstorage in the communication device data storage may be received fromone or more sensors (not shown) external to the communication device 124via the communication device transceiver 302. Alternatively, informationfor storage in the communication device data storage 312 may be receivedfrom one or more communication device sensors 320. For example,tutorials, room layouts, inventory, checklists, and the like may bestored in the communication device data storage 312. Medical personnel130 can create, revise, or refine medical, procedure, and inventor notesas appropriate using the communication device user interface 304 tostore new information in the communication device data storage 312.

The communication device 124 in the exemplary example includes thecommunication device transceiver 302. The communication devicetransceiver 302 incorporating within a communication device transceiverantenna (not shown), enables wireless communication from thecommunication device 124 to, for example, the optimization computingdevice 206 and the network 210 of FIG. 2. It will be appreciated bythose of ordinary skill in the art that the communication device 124 mayinclude a single communication device transceiver 302 as shown, oralternatively separate transmitting and receiving components, forexample but not limited to, a transmitter, a transmitting antenna, areceiver, and a receiving antenna.

The communication device 124 in the illustrated example includes one ormore communication device sensors 320. It will be appreciated by thoseof ordinary skill in the art that the one or more communication devicesensors 320 may be of any sensor technology now known or in the futuredeveloped. For example, the one or more communication device sensors 320may be IoT (Internet of Things) sensors, RFID (radio frequencyidentification) sensors, image sensors, light based (lidar) sensors,biometric sensors, printed sensors, wearable sensors, and optical imagesensors. IoT sensors include temperature sensors, proximity sensors,pressure sensors, RF (radio frequency) sensors, pyroelectric IR(infrared) sensors, water-quality sensors, chemical sensors, smokesensors, gas sensors, liquid-level sensors, automobile sensors andmedical sensors.

Each of the one or more communication device sensors 320 comprise adetector allowing the monitoring and control of various parameterswithin the medical procedure room, for example, environmental parameters(temperature, humidity, carbon dioxide, and the like), technologicalprocesses (automation, robotics, materials analysis, and the like),and/or biometric tracking (movement, health contextual conditions, andthe like). More specifically, the one or more communication devicesensors 320 may provide personal fitness monitoring of a user of thecommunication device 124, a patient, and/or any other personnelassociated with the medical procedure room. Alternatively, the one ormore communication device sensors 320 may provide automation such assecurity, lighting, energy management, and access control for themedical procedure room. Alternatively, the one or more communicationdevice sensors 320 may provide monitoring of the various devices andequipment associated with the medical procedure room. Alternatively, theone or more communication device sensors 320 may provide haptic orproprioception inputs, such as via accelerometers or bionic exoskeletonstyle components, which assess relative position of mechanicalcomponents of a joint or prosthesis or robotic arm, applicator deviceand the like. In operation, the one or more communication device sensors320 communicate with one another, with other sensors within the medicalprocedure room, and/or with any other device within or external to themedical procedure room. For example, although not illustrated, the oneor more communication device sensors 320 may communicate directly orindirectly with sensors implanted within a patient.

FIG. 4 is a block diagram of one exemplary embodiment of a mixed realitydevice 132 for use within the optimization system 104 of FIG. 2 inaccordance with some embodiments. The mixed reality device 132 mayprovide a virtual reality interface in which a computer-simulatedreality electronically replicates an environment with which a user mayinteract. In some embodiments, the mixed reality device 132 may providean augmented reality interface in which a direct or indirect view ofreal-world environments in which the user is currently disposed areaugmented (for example, supplemented, by additional computer-generatedsensory input such as sound, video, images, graphics, Global PositioningSystem (GPS) data, or other information). In still other embodiments,the mixed reality device 132 may provide a mixed reality interface inwhich electronically generated objects are inserted in a direct orindirect view of real-world environments in a manner such that they mayco-exist and interact in real time with the real-world environment andreal-world objects. It will be appreciated by those of ordinary skill inthe art that the mixed reality device 132 may comprise any mixed realityor virtual reality technology now known or in the future developed.

The mixed reality device 132 is electrically and/or communicativelyconnected to a variety of other devices and databases as previouslydescribed with respect to FIG. 2 herein. In some embodiments, the mixedreality device 132 includes a plurality of electrical and electroniccomponents, providing power, operational control, communication, and thelike within the mixed reality device 132. For example, the mixed realitydevice 132 in one embodiment includes, among other things, a mixedreality device transceiver 402, a mixed reality device user interface404, a mixed reality device network interface 406, a mixed realitydevice processor 408, a mixed reality device memory 410, and one or moremixed reality device sensors 424.

It should be appreciated by those of ordinary skill in the art that FIG.4 depicts the mixed reality device 132 in a simplified manner and apractical embodiment may include additional components and suitablyconfigured logic to support known or conventional operating featuresthat are not described in detail herein. It will further be appreciatedby those of ordinary skill in the art that the mixed reality device 132may be a head-mounted display device in the form of eyeglasses, goggles,a helmet, a visor, or any other mixed reality device eyewear now knownor in the future developed. It will further be appreciated by those ofordinary skill in the art that the mixed reality device 132 generatesand/or displays virtual reality images, mixed reality images, and/oraugmented reality images. In the mixed reality device 132, a sceneproduced on a display device can be oriented or modified based on userinput. The mixed reality device 132 provides a visual image in whichreal world and virtual world objects are presented together within asingle display. It will be appreciated by those of ordinary skill in theart that although the embodiments herein are illustrated with a mixedreality device, alternative embodiments within the scope include avirtual reality device or an augmented reality device.

The components of the mixed reality device 132 (for example 402, 404,406, 408 and 410) are communicatively coupled via a mixed reality devicelocal interface 418. The mixed reality device local interface 418 maybe, for example but not limited to, one or more buses or other wired orwireless connections, as is known in the art. The mixed reality devicelocal interface 418 may have additional elements, which are omitted forsimplicity, such as controllers, buffers (caches), drivers, repeaters,and receivers, among many others, to enable communications. Further, themixed reality device local interface 418 may include address, control,and/or data connections to enable appropriate communications among theaforementioned components.

The mixed reality device processor 408 is a hardware device forexecuting software instructions. The mixed reality device processor 408may be any custom made or commercially available processor, a centralprocessing unit (CPU), an auxiliary processor among several processorsassociated with the mixed reality device processor 408, asemiconductor-based microprocessor, or generally any device forexecuting software instructions. When the mixed reality device 132 is inoperation, the mixed reality device processor 408 is configured toexecute software stored within the mixed reality device memory 410, tocommunicate data to and from the mixed reality device memory 410, and togenerally control operations of the mixed reality device 132 pursuant tothe software instructions.

The mixed reality device user interface 404 may be used to receive userinput from and/or for providing system output to the user or to one ormore devices or components. The mixed reality device user interface 404may include one or more input devices, including but not limited to anavigation key, a function key, a microphone, a voice recognitioncomponent, joystick or any other mechanism capable of receiving an inputfrom a user, or any combination thereof. Further, mixed reality deviceuser interface 404 may include one or more output devices, including butnot limited to a speaker, headphones, display, or any other mechanismcapable of presenting an output to a user, or any combination thereof.In some embodiments, the mixed reality device user interface 404includes a user interface mechanism such as a touch interface or gesturedetection mechanism that allows a user to interact with the displayelements of the mixed reality device display 422 or projected into theeyes of the user.

As illustrated, a mixed reality device display 422 may be a separateuser interface or combined within the mixed reality device userinterface 404. The mixed reality device display 422 may provide a twodimensional or three-dimensional image visible to the wearer of themixed reality device 132. The mixed reality device display 422 may be,for example, a projection device for displaying information such astext, images, or video received from the optimization computing device206, communication devices 124, and/or remote connections 208 via thenetwork 210 of FIG. 2.

The mixed reality device user interface 404 may further include, forexample, a serial port, a parallel port, an infrared (IR) interface, auniversal serial bus (USB) interface and/or any other interface hereinknown or in the future developed.

The mixed reality device network interface 406 may be used to enable themixed reality device 132 to communicate on a network, such as thenetwork 210 of FIG. 2, a wireless access network (WAN), a radiofrequency (RF) network, and the like. The mixed reality device networkinterface 406 may include, for example, an Ethernet card or adapter or awireless local area network (WLAN) card or adapter. Additionally, oralternatively the mixed reality device network interface 406 may includea radio frequency interface for wide area communications such asLong-Term Evolution (LTE) networks, or any other network now known or inthe future developed. The mixed reality device network interface 406 mayinclude address, control, and/or data connections to enable appropriatecommunications on the network.

The mixed reality device memory 410 may include any non-transitorymemory elements comprising one or more of volatile memory elements (forexample, random access memory (RAM), nonvolatile memory elements (forexample, read only memory “ROM”), and combinations thereof. Moreover,the mixed reality device memory 410 may incorporate electronic,magnetic, optical, and/or other types of storage media. Note that themixed reality device memory 410 may have a distributed architecture,where various components are situated remotely from one another, but canbe accessed by the mixed reality device processor 408. The software inthe mixed reality device memory 410 may include one or more softwareprograms, each of which includes an ordered listing of executableinstructions for implementing logical functions. The software in themixed reality device memory 410 includes a suitable mixed reality deviceoperating system 414 and one or more mixed reality device applications416. The mixed reality device operating system 414 controls theexecution of other computer programs, such as the one or more mixedreality device applications 416, and provides scheduling, input-outputcontrol, file and data management, memory management, and communicationcontrol and related services. The one or more mixed reality deviceapplications 416 may be configured to implement the various processes,algorithms, methods, techniques, and the like described herein.

The mixed reality device memory 410 further includes a mixed realitydevice data storage 412 used to store data. In the exemplary embodimentof FIG. 4, the mixed reality device data storage 412 is located internalto the mixed reality device memory 410 of the mixed reality device 132.Additionally, or alternatively, (not shown) the mixed reality devicedata storage 412 may be located external to the mixed reality device 132such as, for example, an external hard drive connected to the mixedreality device user interface 404. In a further embodiment, (not shown)the mixed reality device data storage 412 may be located external andconnected to the mixed reality device 132 through a network and accessedvia the mixed reality device network interface 406.

In operation, information for storage in the mixed reality device datastorage 412 may be entered via the mixed reality device user interface404. In some embodiments, the mixed reality device data storage 412stores data received by an augmented reality interface 420 which, forexample, recognizes and registers the spatial characteristics of amedical procedure room.

Alternatively, information for storage in the mixed reality device datastorage 412 may be received from the optimization computing device 206,the communication devices 124, or the remote connections 208 via themixed reality device transceiver 402. Alternatively, information forstorage in the mixed reality device data storage 412 may be receivedfrom one or more sensors (not shown) external to the mixed realitydevice 132 via the mixed reality device transceiver 402. Alternatively,information for storage in the mixed reality device data storage 412 maybe received from one or more mixed reality device sensors 424. Forexample, tutorials, room layouts, inventory, checklists, and the likemay be stored in the mixed reality device data storage 412. Medicalpersonnel 130 can create, revise, or refine medical, procedure, andinventor notes as appropriate using the mixed reality device userinterface 420 to store new information in the mixed reality device datastorage 412.

The mixed reality device 132 in the exemplary example includes the mixedreality device transceiver 402. The mixed reality device transceiver 402incorporating within a mixed reality device transceiver antenna (notshown), enables wireless communication from the mixed reality device 132to, for example, the optimization computing device 206 and the network210 of FIG. 2. It will be appreciated by those of ordinary skill in theart that the mixed reality device 132 may include a single communicationdevice transceiver 302 as shown, or alternatively separate transmittingand receiving components, for example but not limited to, a transmitter,a transmitting antenna, a receiver, and a receiving antenna.

The mixed reality device 132 in the illustrated example includes one ormore mixed reality device sensors 424. It will be appreciated by thoseof ordinary skill in the art that the one or more mixed reality devicesensors 424 may be of any sensor technology now known or in the futuredeveloped. For example, the one or more mixed reality device sensors 424may be IoT (Internet of Things) sensors, RFID (radio frequencyidentification) sensors, image sensors, light based (lidar) sensors,biometric sensors, printed sensors, wearable sensors, and optical imagesensors. IoT sensors include temperature sensors, proximity sensors,pressure sensors, RF (radio frequency) sensors, pyroelectric IR(infrared) sensors, water-quality sensors, chemical sensors, smokesensors, gas sensors, liquid-level sensors, automobile sensors andmedical sensors.

Each of the one or more mixed reality device sensors 424 comprise adetector allowing the monitoring and control of various parameterswithin the medical procedure room, for example, environmental parameters(temperature, humidity, carbon dioxide, and the like), technologicalprocesses (automation, robotics, materials analysis, and the like),and/or biometric tracking (movement, health contextual conditions, andthe like). More specifically, the one or more mixed reality devicesensors 424 may provide personal fitness monitoring of a user of themixed reality device 132, a patient, and/or any other personnelassociated with the medical procedure room. Alternatively, the one ormore mixed reality device sensors 424 may provide automation such assecurity, lighting, energy management, and access control for themedical procedure room. Alternatively, the one or more mixed realitydevice sensors 424 may provide monitoring of the various devices andequipment associated with the medical procedure room. Alternatively, theone or more mixed reality device sensors 424 may provide haptic orproprioception inputs, such as via accelerometers or bionic exoskeletonstyle components, which assess relative position of mechanicalcomponents of a joint or prosthesis or robotic arm, applicator deviceand the like. In operation, the one or more mixed reality device sensors424 communicate with one another, with other sensors within the medicalprocedure room, and/or with any other device within or external to themedical procedure room.

FIG. 5 is a block diagram of one exemplary embodiment of an optimizationcomputing device 206 for use within the optimization system 104 of FIG.2. Specifically, the optimization computing device 206 can implement thevarious methods described herein.

The optimization computing device 206 is electrically and/orcommunicatively connected to a variety of other devices and databases aspreviously described with respect to FIG. 2 herein. In some embodiments,the optimization computing device 206 includes a plurality of electricaland electronic components, providing power, operational control,communication, and the like within the optimization computing device206. For example, the optimization computing device 206 in oneembodiment includes, among other things, an optimization computingdevice transceiver 502, an optimization computing device user interface504, an optimization computing device network interface 506, anoptimization computing device processor 508, an optimization computingdevice memory 510, and one or more optimization computing devicesensor(s) 522.

It should be appreciated by those of ordinary skill in the art that FIG.5 depicts the optimization computing device 206 in a simplified mannerand a practical embodiment may include additional components andsuitably configured logic to support known or conventional operatingfeatures that are not described in detail herein. It will further beappreciated by those of ordinary skill in the art that the optimizationcomputing device 206 may be a personal computer, desktop computer,tablet, smartphone, or any other computing device now known or in thefuture developed.

It will further be appreciated by those of ordinary skill in the artthat the optimization computing device 206 alternatively may functionwithin a remote server, cloud computing device, or any other remotecomputing mechanism now known or in the future developed. For example,the optimization computing device 206 in some embodiments may be a cloudenvironment incorporating the operations of the optimization computingdevice processor 508, the optimization computing device memory 510, theoptimization computing device user interface 504, and various otheroperating modules to serve as a software as a service model for thecommunication devices 124 and the mixed reality devices 132.

The components of the optimization computing device 206 (for example502, 504, 506, 508 and 510) are communicatively coupled via anoptimization computing device local interface 518. The optimizationcomputing device local interface 518 may be, for example but not limitedto, one or more buses or other wired or wireless connections, as isknown in the art. The optimization computing device local interface 518may have additional elements, which are omitted for simplicity, such ascontrollers, buffers (caches), drivers, repeaters, and receivers, amongmany others, to enable communications. Further, the optimizationcomputing device local interface 518 may include address, control,and/or data connections to enable appropriate communications among theaforementioned components.

The optimization computing device processor 508 is a hardware device forexecuting software instructions. The optimization computing deviceprocessor 508 may be any custom made or commercially availableprocessor, a central processing unit (CPU), an auxiliary processor amongseveral processors associated with the optimization computing deviceprocessor 508, a semiconductor-based microprocessor, or generally anydevice for executing software instructions. When the optimizationcomputing device 206 is in operation, the optimization computing deviceprocessor 508 is configured to execute software stored within theoptimization computing device memory 510, to communicate data to andfrom the optimization computing device memory 510, and to generallycontrol operations of the optimization computing device 206 pursuant tothe software instructions.

The optimization computing device user interface 504 may be used toreceive user input from and/or for providing system output to the useror to one or more devices or components. User input may be provided via,for example, a keyboard, touch pad, and/or a mouse. System output may beprovided via a display device, speakers, and/or a printer (not shown).The optimization computing device user interface 504 may furtherinclude, for example, a serial port, a parallel port, an infrared (IR)interface, a universal serial bus (USB) interface and/or any otherinterface herein known or in the future developed.

The optimization computing device network interface 506 may be used toenable the optimization computing device 206 to communicate on anetwork, such as the network 210 of FIG. 2, a wireless access network(WAN), a radio frequency (RF) network, and the like. The optimizationcomputing device network interface 506 may include, for example, anEthernet card or adapter or a wireless local area network (WLAN) card oradapter. Additionally, or alternatively the optimization computingdevice network interface 506 may include a radio frequency interface forwide area communications such as Long-Term Evolution (LTE) networks, orany other network now known or in the future developed. The optimizationcomputing device network interface 506 may include address, control,and/or data connections to enable appropriate communications on thenetwork.

The optimization computing device memory 510 may include anynon-transitory memory elements comprising one or more of volatile memoryelements (for example, random access memory (RAM), nonvolatile memoryelements (for example, read only memory “ROM”), and combinationsthereof. Moreover, the optimization computing device memory 510 mayincorporate electronic, magnetic, optical, and/or other types of storagemedia. Note that the optimization computing device memory 510 may have adistributed architecture, where various components are situated remotelyfrom one another, but can be accessed by the optimization computingdevice processor 508. The software in the optimization computing devicememory 510 may include one or more software programs, each of whichincludes an ordered listing of executable instructions for implementinglogical functions. The software in the optimization computing devicememory 510 includes a suitable optimization computing device operatingsystem 514 and optimization programming code 512. The optimizationcomputing device operating system 514 controls the execution of othercomputer programs, such as the optimization program code 512, andprovides scheduling, input-output control, file and data management,memory management, and communication control and related services. Theoptimization program code 512 may be configured to implement the variousprocesses, algorithms, methods, techniques, and the like describedherein.

The optimization computing device memory 510 further includes anoptimization computing device data storage 516 used to store data. Inthe exemplary embodiment of FIG. 5, the optimization computing devicedata storage 516 is located internal to the optimization computingdevice memory 510 of the optimization computing device 206.Additionally, or alternatively, (not shown) the optimization computingdevice data storage 516 may be located external to the optimizationcomputing device 206 such as, for example, an external hard driveconnected to the optimization computing device user interface 504. In afurther embodiment, (not shown) the optimization computing device datastorage 516 may be located external and connected to the optimizationcomputing device 206 through a network and accessed via the optimizationcomputing device network interface 506.

The optimization computing device data storage 516, in accordance withsome embodiments, stores optimization data 520 for operational use inthe various processes, algorithms, methods, techniques, and the likedescribed herein. In operation, information for storage in theoptimization computing device data storage 516 may be entered via theoptimization computing device user interface 504. Alternatively,information for storage in the optimization computing device datastorage 516 may be received from the mixed reality devices 132, thecommunication devices 124, or the remote connections 208 via theoptimization computing device transceiver 502. Alternatively,information for storage in the optimization computing device datastorage 516 may be received from one or more sensors (not shown)external to the optimization computing device 206 via the optimizationcomputing device transceiver 502. Alternatively, information for storagein the optimization computing device data storage 516 may be receivedfrom one or more optimization computing device sensors 522. For example,tutorials, room layouts, inventory, checklists, and the like may bestored in the optimization computing device data storage 516. Medicalpersonnel 130 can create, revise, or refine medical, procedure, andinventor notes as appropriate using the optimization computing deviceuser interface 504 to store new information in the optimizationcomputing device data storage 516.

The optimization computing device 206 in the exemplary example includesthe optimization computing device transceiver 502. The optimizationcomputing device transceiver 502 incorporating within an optimizationcomputing device transceiver antenna (not shown), enables wirelesscommunication from the optimization computing device 206 to, forexample, one or more communication devices 124, one or more mixedreality devices 132, and the network 210. It will be appreciated bythose of ordinary skill in the art that the optimization computingdevice 206 may include a single optimization computing devicetransceiver as shown, or alternatively separate transmitting andreceiving components, for example but not limited to, a transmitter, atransmitting antenna, a receiver, and a receiving antenna.

The optimization computing device 206 in the illustrated exampleincludes one or more optimization computing device sensors 522. Each ofthe one or more optimization computing device sensors 522 comprise adetector allowing the monitoring and control of various parameterswithin the medical procedure room, for example, environmental parameters(temperature, humidity, carbon dioxide, and the like), technologicalprocesses (automation, robotics, materials analysis, and the like),and/or biometric tracking (movement, health contextual conditions, andthe like). More specifically, the one or more optimization computingdevice sensors 522 may provide personal fitness monitoring of a user ofthe optimization computing device 206, a patient, and/or any otherpersonnel associated with the medical procedure room. Alternatively, theone or more optimization computing device sensors 522 may provideautomation such as security, lighting, energy management, and accesscontrol for the medical procedure room. Alternatively, the one or moreoptimization computing device sensors 522 may provide monitoring of thevarious devices and equipment associated with the medical procedureroom. Alternatively, the one or more optimization computing devicesensors 522 may provide haptic or proprioception inputs, such as viaaccelerometers or bionic exoskeleton style components, which assessrelative position of mechanical components of a joint or prosthesis orrobotic arm, applicator device and the like. In operation, the one ormore optimization computing device sensors 522 communicate with oneanother, with other sensors within the medical procedure room, and/orwith any other device within or external to the medical procedure room.

FIG. 6 is a flow diagram of a method for medical procedure optimizationin accordance with some embodiments. Specifically, FIG. 6 is a flowdiagram for an initial setup program 600 of a medical procedureoptimization in accordance with some embodiments. The initial setupprogram 600, for example, may be implemented within the optimizationprogram code 512 of FIG. 5. In an alternative embodiment, the initialsetup program 600 may be implemented as a cloud-based internet programaccessed via the communication devices 124 and the optimizationcomputing device 206. In yet another alternative embodiment, the initialsetup program 600 can be distributively implemented within a system inwhich the various components are remotely located from each other inother embodiments. For example, a first set of components of the initialsetup program 600 may be implemented and stored within the optimizationcomputing device 206, a second set of components of the initial setupprogram 600 may be implemented and stored within one or more of thecommunication devices 124, a third set of components of the initialsetup program 600 may be implemented and stored within one or more ofthe mixed reality devices 132, and/or a fourth set of components of theinitial setup program 600 may be implemented and stored within otherdevices connected to the network 210 or otherwise communicativelycoupled to the optimization computing device 206, the communicationdevices 124, and the mixed reality devices 132. It will be appreciatedthat any and all distribution arrangements of the initial setup program600 are within the scope of the claimed invention herein.

It will be appreciated by those of ordinary skill in the art that theflow diagram of FIG. 6 is simply an exemplary embodiment and otheralternative process flows are within the scope of the claimed inventionherein.

In operation, the optimization computing device processor 508 accessesand executes the initial setup program 600. As illustrated in FIG. 6,the initial setup program 600 begins with the receipt of various inputsand information to process an initial medical procedure setup. Forexample, the optimization computing device 206 receives user input atits optimization computing device user interface 504, stores theinformation within the user input in the optimization computing devicedata storage 516, and accesses the optimization program code 512 by theoptimization computing device processor 508 for executing initial setupprogram 600. Alternatively, the communication device 124 receives userinput including setup information at its communication device userinterface 304, sends the information via its communication devicenetwork interface 306 through the network 210 to the optimizationcomputing device 206. The optimization computing device 206 thereafterreceives the information via its optimization computing device networkinterface 506, stores the information within the user input in theoptimization computing device data storage 516, and accesses theoptimization program code 512 by the optimization computing deviceprocessor 508 for executing initial setup program 600. It will beappreciated that the information may originate from and be receivedthrough various alternative methods in accordance with some embodiments.

Referring to FIG. 6, the initial setup program 600 begins generally witha new profile at operation 602 including creating a profile at operation604 followed by email confirmation at operation 606. The initial setupprogram 600 thereafter proceeds to or alternatively begins with a loginat operation 608. Next, in operation 610 a creator portal opens anddisplays. Next, in one embodiment, an edit to the medical procedure roomoccurs at operation 612. For example, the edit to the medical procedureroom includes one or more of change in the available room layout anddimensions, change in position of the items corresponding to one or moreof the medical devices, consumables, general tray, and equipment in themedical procedure room. Thereafter, the edited room is saved atoperation 614. Alternatively, from operation 610, a new medicalprocedure room is created at operation 616. For instance, the newmedical procedure room is created by providing required layout anddimensions for the new medical procedure room. It will be appreciatedthat the required layout and dimensions can be provided by using varioustechniques, such as but not limited to, uploading images of the medicalprocedure room. In some embodiments, the created new medical procedureroom or the edited room is assigned a medical procedure room identifier.Thereafter, or after operation 614, the medical practitioner input isreceived and saved in operation 618. In accordance with variousembodiments, the medical practitioner input may include the medicalpractitioner identifier, such as but not limited to, the name, the ID,or the like of the medical practitioner. Next, a procedure input isreceived and saved in operation 620. In accordance with variousembodiments, the procedure input may include the procedure identifier,such as but not limited to, the name, the ID, or the like of the medicalprocedure. In operation 622 a room canvas is created. In accordance withvarious embodiments, the room canvas is created based on previousmedical procedure room setups that are associated with the receivedmedical practitioner input, the procedure input, or the edited/newmedical room. The room canvas of operation 622 next includes creatingconsumables in operation 624, creating medical devices in operation 625,creating a general tray in operation 626 and creating equipment lists inoperation 628.

It will be appreciated that in some embodiments, along with theinformation of room canvas created in 622, the data entered inoperations 628, 624, 625, and 626 precisely configures a given medicalpractitioner's room organization for a given procedure. For example, thedata entered in operations 628, 624, 625, and 626, in some embodiments,identifies inventory of fixed equipment, external enabling technologiessuch as microscopes, drills, fluoroscopy units, navigation systems,robotic systems, lights, electrophysiology systems, anesthetic systems,vacuum and gas management and operating room back table. Similarly, thedata entered in operations 628, 624, 625, and 626, in some embodiments,identifies surgical instruments/items, reusable and disposable. Forexample, the data entered in operations 628, 624, 625, and 626, in someembodiments, identifies medical devices including pharmaceuticaldevices, implants, generators, stimulators, screws, plates, cages,arthroplasty joint replacement devices, heart valves, stents, coils,pacemakers, portals, biologics, catheters, and shunts. Similarly, thedata entered in operations 628, 624, 625, and 626, in some embodiments,identifies chargeable resources/items such as sutures, sponges, clips,medical implants, screws, rods, arthroplasty devices, stimulators,needles, scalpel blades, and drill bits. Similarly, the data entered inoperations 628, 624, 625, and 626, in some embodiments, identifiespharmaceutical resources such as medications, anesthetics, antibiotics,cardiac drugs, blood pressure drugs, sedatives, paralytic agents, painmanagement, and the like.

Continuing with FIG. 6, thereafter, in operation 630 all the itemscorresponding to the medical devices, consumables, general tray, andequipment from the previous one or more operations, or the previousmedical procedures are accessed from a searchable index/list inoperation 630. For example, in some embodiments, preference cards foreach medical practitioner's preferences and procedures are preloaded foraccess during the initial setup procedure 600.

Next, in operation 632, the items are populated in operation 632. Forexample, in one embodiment the items are populated using a drag and dropmethod. In an embodiment, the data associated with usage, such as butnot limited to, sequence of use, pattern of use, instructions for use,priority of use of one or more items and/or equipment during the medicalprocedure is also provided. It will be appreciated that any suitablemethod can be used for operation 632. Next in operation 634, the itemsare placed in the medical procedure room 634 virtually. Thereafter, theplacement is confirmed in operation 636 or alternatively, edits arecompleted in operation 640 and cycled back through operations 630, 632,and 634 until confirmation in operation 636 occurs.

Lastly, in operation 638, the room configuration, data and all otherinformation associated with the medical practitioner for the particularprocedure are stored. It will be appreciated, that the roomconfiguration, data, and information may be stored within one or more ofthe optimization computing device memory 510, one or more of thecommunication device memory 310, one or more of the mixed reality devicememory 410 or any combination thereof.

Upon completion of the initial setup program 600, the stored medicalpractitioner specific and procedure specific room configuration, data,and information would provide medical personnel 130 whose responsibilityis to organize, prepare and set up a medical procedure space withspecific detail to best organize, optimize and prepare the space forefficient and predictable execution of that procedure for that medicalpractitioner. It will be further appreciated that, in some embodiments,although not illustrated in FIG. 6, additional patient specificinformation may be entered and stored. It will further be appreciatedthat the initial setup created and stored can be used hereinafter tocreate a three-dimensional mixed reality map of the medical procedureroom 102 with items, tools, consumables, and equipment placement basedon medical practitioner and procedure preference.

FIG. 7 is a flow diagram of a method for medical procedure optimizationin accordance with some embodiments. Specifically, FIG. 7 is a flowdiagram of a medical procedure initial setup 700. The medical procedureinitial setup 700, for example, may be implemented within the mixedreality device applications 416 of FIG. 4. In an alternative embodiment,the medical procedure initial setup 700 may be implemented as acloud-based internet program accessed by one or more of the mixedreality devices 132. In yet another alternative embodiment, the medicalprocedure initial setup 700 can be distributively implemented within asystem in which the various components are remotely located from eachother in other embodiments and accessed by one or more of the mixedreality devices 132. It will be appreciated that any and alldistribution arrangements of the medical procedure initial setup 700 arewithin the scope of the claimed invention herein.

It will be appreciated by those of ordinary skill in the art that theflow diagram of FIG. 7 is simply an exemplary embodiment and otheralternative process flows are within the scope of the claimed inventionherein.

In operation, the mixed reality device processor 408 accesses andexecutes the medical procedure initial setup 700. As illustrated in FIG.7, the medical procedure initial setup 700 begins with the receipt ofvarious inputs and information to process an initial medical proceduresetup 106. For example, the mixed reality device 132 receives user inputat its mixed reality device user interface 404, stores the informationwithin the user input in the mixed reality device data storage 412, andaccesses the mixed reality device applications 416 by the mixed realitydevice processor 408 for executing the medical procedure initial setup700. Alternatively, the communication device 124 receives user inputincluding setup information at its communication device user interface304, sends the information via its communication device networkinterface 306 through the network 210 to the mixed reality device 132.The mixed reality device 132 thereafter receives the information via itsmixed reality device network interface 406, stores the informationwithin the user input in the mixed reality device data storage 412, andaccesses the mixed reality device applications 416 by the mixed realitydevice processor 408 for executing the medical procedure initial setup700. It will be appreciated that the information may originate from andbe received through various alternative methods in accordance with someembodiments.

Referring to FIG. 7, the medical procedure initial setup 700 beginsgenerally with a new profile at operation 702 including creating aprofile at operation 704 followed by email confirmation at operation706. The medical procedure initial setup 700 thereafter proceeds to oralternatively begins with a login at operation 708. Next, in operation710 an experience portal opens and displays. In some embodiments, theexperience portal is displayed on the mixed reality device display 422,providing, for example, a mixed reality representation of the medicalprocedure environment.

Next, in operation 712, a medical practitioner identifier (for example amedical practitioner) is selected from a list of medical practitioneridentifiers. Next, in operation 714 a procedure identifier is selectedfrom a list of procedure identifiers. Once the medical practitioneridentifier and the procedure identifier have been selected, a medicalprocedure room 102 is identified. For example, in one embodiment themedical procedure room identifier is obtained by scanning a QuickResponse code (“QR code”) representing the medical procedure room 102 atoperation 716.

With knowledge of the medical practitioner, the procedure, and themedical procedure room, the operation next auto places consumables inoperation 718, auto places medical devices in operation 719, auto placesthe general tray in operation 720, auto places the equipment in 722,along with any auto placements related to the medical procedure room,procedure, and medical practitioner. It will be appreciated by those ofordinary skill in the art that any and all tangible items now known orlater discovered for use in the particular procedure may be auto placedby the method 700. As previously described herein, the medicalpractitioner, procedure, and associated auto placements may be stored inone or more of the mixed reality device data storage 412, thecommunication device data storage 312, the optimization computing devicedata storage 516, a cloud-based memory accessed by one or more of themixed reality devices 132, or any other associated memorycommunicatively coupled to the mixed reality device 132. In other words,representations in visual mixed reality format specific to a givenmedical practitioner for a given procedure may be stored in one or moredata storage devices. In this manner, the medical procedure initialsetup 700 may provide preference cards for each medical practitioner'spreferences and procedures pre-loaded for predictive planning andeducation on each procedure step by step from start to finish.

The medical procedure initial setup 700 continues with operation 730wherein a medical procedure room map is rendered. For example, the roommap may be rendered on the mixed reality device display 422 for visualaccess. Thereafter, in operation 738 a searchable index is accessed, andlastly in operation 740 the tool/tray is highlighted. For example, inone embodiment, the inputted data would be mixed reality expressed byone or more of mixed reality interface augmented realityglasses/goggles/headpieces.

The previously described operations of the medical procedure initialsetup 700 is completed during a setup time 724. Further, a medicalprocedure 732 is completed during a procedure time 726. In accordancewith various embodiments, the data related to usage of one or more item,equipment, or device is acquired during the procedure time 726. Lastlyan implant count 734 and a consumable count 736 are identified andstored during a transitional time 728.

The medical procedure initial setup 700 provides for configuration,organization and three-dimensional planning of the geometric space ofthe medical procedure room environment for a given medical practitionerfor a given procedure. Specifically, the mixed reality device display,for example, a holograph glass system, may create a virtualthree-dimensional environment for the entire operating organizationfield for a specific medical practitioner, and a specific procedure andfor a specific patient, enabling complete data control and digitallymanaged integration of data of equipment for each specific procedure foreach specific patient, with extreme precision of inventory use, control,management, performance, tracking and billing sales control andaccuracy.

In one example, once the medical procedure initial setup 700 iscompleted, a medical personnel 130, such as a surgical technician, wouldbe able to look through the glasses and see mixed reality representationof each medical instrument, device or inventory item, and identifyexactly where on the field it would be placed, what number of deviceswould be placed, what the device was called, and ultimately provide forinstrument or device placement, identification, tracking registrationand analytics, and ultimately inventory management. Disposableinstruments or medical implants/devices would be able to be logged infor inventory at this step and converted to charge after application oruse at the end of the procedure.

FIG. 8 is a functional block diagram of a system of medical procedureoptimization in accordance with some embodiments. Specifically, FIG. 8illustrates an optimization functional block diagram 800 utilizingartificial intelligence in a continuous loop system of optimization inaccordance with some embodiments. The optimization functional blockdiagram 800, in some embodiments, may be implemented within theoptimization system 104 as described previously herein. Specifically,the optimization functional block diagram 800, in some embodiments,illustrates the optimization of preparation of a procedure, steps andorder of steps of a procedure, orientation of medical personnel 130 fora procedure, and location and identification of equipment for aprocedure. The real-time feedback and training of the optimizationfunctional block diagram 800 allows for optimized efficiency andreduction of unnecessary steps throughout medical procedures. It will beappreciated that the optimization functional block diagram 800 inoperation enables identification of experience based predictive data forperformance, safety, outcome efficiency and inventory control/managementfor one or more procedures.

Module 802 implements a medical procedure set up. For example, inoperation, the module 802 may implement the medical procedure set up asdescribed hereinbefore for FIGS. 1 through 7. In some embodiments, themodule 802 is configured to obtain at least one medical procedure inputassociated with one or more of a medical practitioner identifier, amedical procedure identifier, and a medical procedure room identifierfor execution of the medical procedure. The medical procedure input mayinclude one or more of steps for execution of the medical procedure,order of steps for execution of the medical procedure, and instructionsfor one or more steps for execution of the medical procedure. In someembodiments, the medical procedure setup input may include one or moreof the medical practitioner identifier, the medical procedureidentifier, and the medical procedure room identifier.

In some embodiments, the procedure set up of module 802 is implementedfor a specific medical practitioner's preference for a particularprocedure in a particular medical procedure room 102. For example, thedata stored in the data module 810 may include pre-stored preferenceinformation for each medical practitioner's preferences and procedures.Medical personnel 130, in operation, utilize the medical practitionerspecific, procedure specific, room specific data stored in the datamodule 810 to organize, prepare and set up a medical procedure withspecific detail to best organize, optimize and prepare for efficient andpredictable execution of that procedure. It will be appreciated that themedical practitioner specific, procedure specific, room specific datastored in the data module 810 allows for medical practitioners andmedical personnel 130 to best organize and optimize efficient andpredictable execution of a procedure. For example, the procedure setupmay include a layout and orientation of where each medical practitionerand medical procedure room personnel will be positioned during themedical procedure. Further, the procedure set up may include detailedsteps and instructions for each step of the procedure process.

Information from the procedure setup module 802 feeds into the datamodule 810 and also feeds into a procedure input module 804. In someembodiments, the module 802 is configured to provide virtual imageguidance, via one or more mixed reality devices, for execution of themedical procedure based on the received at least one medical procedureinput. In an embodiment, the virtual guidance is provided via athree-dimensional reality map reflecting one or more of the steps forexecution of the medical procedure, the order of steps for execution ofthe medical procedure, and the instructions for one or more steps forexecution of the medical procedure in accordance with the receivedinput. In operation, the procedure set up of module 802 includesutilization of one or more mixed reality devices 132. The one or moremixed reality devices 132 provide the ability to use virtual imageguidance for procedure set up and execution. In some embodiments, theone or more mixed reality devices 132 provide a work-flow chart for agiven procedure, for a given medical practitioner, providing aninteractive real time checklist for execution of that procedure,registering outline, order, sequence, steps completed, steps pending,and organization management for execution of that procedure, andultimately provide data for optimization of future evolutions ofprocedural performance.

In some embodiments, the one or more mixed reality devices 132 provideintegration of multimedia data management, acquisition and expressionfrom secondary enabling medical technologies, such as operatingmicroscope, fluoroscope, navigational system, robotic system, endoscopicsystem, for that medical practitioner, for that procedure for thepurpose of audiovisual data management and expression, given patientconfidentiality standards.

In some embodiments, the one or more mixed reality devices 132 furtherproduce customizable three-dimensional templates for training other teammembers for optimal procedure set ups that reduce unnecessary timewasted.

The one or more mixed reality devices 132 further may provide forinstrument tray recognition and back table recognition for pre-plannedprocedure tray placement reducing setup times and turnover times betweenand during medical procedures. The one or more mixed reality devices 132may also provide for instrument tray three-dimensional view of tray andexpansion ability in order to picture virtual images of all instrumentsinside of tray without opening. In an embodiment, the instruments areNetwork Intelligent Operating Room Equipment having the ability fortracking, counting virtually through object recognition with names andpurpose with detailed explanations. Tracking instruments in this waywill provide less personnel training and provide a guide for limitedpassing throughout procedures. Using pre-populated lists stored in thedata module 810 of exact instruments needed for each specific medicalprocedure and specific location provides for most efficient and leanpractices. The one or more mixed reality devices 132 allowsinstrumentation to be viewed at all angles to become more familiar withfunctionality and how each tool works and assembles as well asdissembles.

The data stored in data module 810 may include, but is not limited to,medical procedure room organization, setup, logistics, performance,inventory and the like. Specifically, the data stored in data module 810includes one or more of the data stored in the communication device datastorage 312, the mixed reality device data storage 412, and theoptimization data 520 stored in the optimization computing device datastorage 516, all as described previously herein.

The data stored in the data module 810, may also include, for example,technique guides on instrumentation and various equipment used for eachprocedure preloaded for real-time feedback and training that willoptimize efficiency and reduce unnecessary steps throughout proceduresthat will overall provide a safer environment to patient care. The datastored in the data module 810, may also include, for example, tutorialeducation for each instrument and each medical procedure tray by nameand its purpose for use in medical procedure for real-time feedback,support or training. It will be appreciated that stored video tutorialsfor anatomy and physiology for a medical procedure will reduce time intraining personnel for new procedures and cut down on redundancy andoverall reduce risk factors and improve safety for patient care andhealthcare providers. In general, the data stored in the data module 810provides for predictive planning and education on each procedure step bystep from start to finish. In an exemplary embodiment, the data module810 is configured to store one or more of technique guides or tutorialsassociated with the execution of one or more steps of the medicalprocedure. The procedure module 802 is further configured to providevirtual image guidance for execution of the medical procedure using thestored tutorials.

The data stored in the data module 810, may also include, for example,guided imaging for setup of the various components of the medicalprocedure room 102 as described previously herein in FIG. 1 forcontinuous updating, editing, and ultimately optimizing. The data storedin the data module 810, may also include, for example, primary andancillary supplies, primary and ancillary equipment listed specificallyto each procedure. In accordance with various embodiments, the datamodule 810 is configured to store one or more steps of one or morepreviously executed medical procedures (herein interchangeably referredto as reference medical procedures) associated with each of the medicalpractitioner identifier, a medical procedure identifier, and a medicalprocedure room identifier for execution of the medical procedure. Forexample, one or more medical procedures previously performed by amedical practitioner is recorded and associated with the medicalpractitioner identifier and stored in the data module 810. Similarly,one or more medical procedures performed in a medical procedure room isrecorded and associated with the medical procedure room identifier andstored in the data module 810. Similarly, each of the one or morepreviously performed medical procedures is recorded and associated witha medical procedure identifier and stored in the data module 810. Insome embodiments, the one or more previous medical procedures may beobtained from the tutorials and technique guides stored in the datamodule 810. In an embodiment, the data module 810 is also configured tostore data, such as, patient name, patient identification, medicalreports, medical diagnosis, health conditions, or the like, associatedwith a patient undergoing the medical procedure.

The data stored in the data module 810 may provide medical personnel 130with the ability to search for instruments to provide information forunfamiliar tools and give feedback of optimal placement. The data storedin the data module 810, may also include, for example, layout ofequipment placement, relative to patients position for a medicalprocedure that will save on turnover times and efficiency for eachprocedure and practitioner preference. The layout and orientation mayinclude where each of the medical practitioners and medical personnel130 will be positioned during the procedure.

In operation, after the procedure room setup module 802, theoptimization functional block diagram 800 proceeds to the procedureinput module 804. The procedure input module 804 includes, but is notlimited to, user input received by one or more of the communicationdevice user interface 304, the communication device sensor(s) 320, themixed reality device user interface 404, the mixed reality devicesensor(s) 424, the optimization computing device user interface 504, andthe optimization computing device sensor(s) 522. It will be appreciatedby those of ordinary skill in the art that the user input may bereceived before, during, and after a particular medical procedure.

In practice, the procedure input module 804 incorporates notes for eachmedical procedure, medical practitioner specific to each procedure forfuture efficiencies and personnel. The procedure input module 804 mayalso incorporate for example usage of one or more primary and ancillarydevices and other medical items including disposable items. In oneembodiment, these items include pre-bar code or reference code allowingto track disposable costs and reduce the amount of wasted procedurecosts.

In an embodiment, the procedure input module 804 is configured toacquire, via the one or more mixed reality devices 132, data associatedwith usage of at least one of an item and an equipment during themedical procedure. For instance, the data associated with usage of atleast one of an item and an equipment includes data associated with atleast a pattern of use, a sequence of use, and a priority of use of atleast one of the item and the equipment during the medical procedure. Inan embodiment, the procedure input module 804, via the one or more mixedreality devices 132, is configured to acquire data associated with atleast one of a position and an orientation of at least one personnel forexecution of the medical procedure. Further, in practice the procedureinput module 804 includes tracking of use (including use in visual mixedreality format), patterns for use, sequence of use, percentage andpriority of use for chargeable resources such as sutures, sponges,clips, medical implants, screws, rods, arthroplasty devices,stimulators, needles, scalpel blades, drill bits.

In some embodiments, the procedure input module 804 is configured toprovide the received data to a machine learning module 812. The machinelearning module 812 may be any system configured to learn and adaptitself to do better in changing environments. The machine learningmodule 812 may employ any one or combination of the followingcomputational techniques: neural network, constraint program, fuzzylogic, classification, conventional artificial intelligence, symbolicmanipulation, fuzzy set theory, evolutionary computation, cybernetics,data mining, approximate reasoning, derivative-free optimization,decision trees, and/or soft computing.

The machine learning module 812 may implement an iterative learningprocess. The learning may be based on a wide variety of learning rulesor training algorithms. The learning rules may include one or more ofback-propagation, patter-by-pattern learning, supervised learning,and/or interpolation. As a result of the learning, the machine learningmodule 812 may learn to determine the operations being performed by theoptimization system 104.

The machine learning module 812 is configured to implement one or moremachine learning algorithms to determine an optimal relationship of thedata associated with usage of at least one of the item and the equipmentin the one or more reference medical procedures with the one or more ofthe medical practitioner identifier, the medical procedure identifier,and the medical procedure room identifier.

In accordance with some embodiments of the invention, the machinelearning algorithm may utilize any machine learning methodology, nowknown or in the future developed, for classification. For example, themachine learning methodology utilized may be one or a combination of:Linear Classifiers (Logistic Regression, Naive Bayes Classifier);Nearest Neighbor; Support Vector Machines; Decision Trees; BoostedTrees; Random Forest; and/or Neural Networks. The machine learningmodule 812 continually evolves the specifics of execution of a medicalprocedure in real time with new data inputs. The machine learning intentis to continually implement optimized medical procedure overtime.

In an embodiment, the machine learning module 812 is configured toanalyze the prestored data, in the data module 810, associated withusage of at least one of the item and the equipment in one or morereference medical procedures corresponding to the one or more of themedical practitioner identifier, the medical procedure identifier, andthe medical procedure room identifier. For example, the prestored dataassociated with usage of at least one of the item and the equipment inone or more reference medical procedures include pattern of use,sequence of use, priority of user of the item and/or the equipmentduring execution of one or more steps of the one or more referencemedical procedures. In an embodiment, the analysis of the prestored datacomprises determining the optimal relationship of the data associatedwith usage of at least one of the item and the equipment in one or morereference medical procedures with the one or more of the medicalpractitioner identifier, the medical procedure identifier, and themedical procedure room identifier. For example, the machine learningmodule 812 analyzes the prestored data in one or more reference medicalprocedures and determines that, for example, the medical personnel 130prefer to apply a specific bandage after applying stitches with the helpof a surgical suture. The machine learning module 812 establishes thisrelationship between the sequence of steps like application of aspecific bandage after performing stitching of the surgical area, andthe medical personnel identifier and stores the relationship data in thedata module 810.

In various embodiments, the machine learning module 812 is configured tocompare the acquired data associated with usage of at least one of theitem and the equipment during the medical procedure with the analyzedprestored data associated with the respective at least one of the itemand the equipment in one or more reference medical procedures. In anembodiment, the machine learning module 812 is configured to compare theacquired data with the analyzed prestored data by determining whetherthe acquired data associated with usage of at least one of the item andthe equipment during the medical procedure correspond to the determinedoptimal relationship for the one or more of the medical practitioneridentifier, the medical procedure identifier, and the medical procedureroom identifier. In the example stated above, at this stage, the machinelearning module 812 determines that whether the specific bandage isapplied after stitching the surgical area with the surgical sutureduring the medical procedure.

The machine learning module 812 is configured to provide recommendationto the module 802 for optimizing the medical procedure based on thecomparison. In accordance with various embodiments, the machine learningmodule 812 is configured to provide recommendation to the module 802 foroptimizing one or more steps of the medical procedure when the acquireddata does not correspond to the determined optimal relationship. Themachine learning module 812 is configured to provide the recommendationsbased on the determined optimal relationship. The recommendations mayinclude one or more of changing the pattern of use, the sequence of use,and the priority of use of at least one of the item and the equipmentduring the medical procedure. For example, in the above scenario, atthis stage when the specific bandage is not applied, after the stitchingis performed then the machine learning module 812 will providerecommendations to apply the specific bandage after stitching. In anembodiment, the procedure module 802 is further configured to update thevirtual image guidance for the medical procedure based on the receivedrecommendations. In an embodiment, the machine learning module 812 isfurther configured to provide the at least one medical procedure inputcorresponding to one or more of the medical practitioner identifier, themedical procedure identifier, and the medical procedure room identifierfor execution of the medical procedure to the procedure module 802.

In an embodiment, the machine learning module 812 is configured toidentify a current step of the medical procedure that is being executedbased on the comparison of the acquired data associated with usage of atleast one of the item and the equipment during the medical procedurewith the analyzed prestored data associated with the respective at leastone of the item and the equipment in one or more reference medicalprocedures. The machine learning module 812 is further configured totransmit, via the procedure input module 804, a notification to at leastone device identifying the current step of the medical procedure. Forexample, when medical personnel 130 is applying bandage to the stitchesof the patient in the medical procedure room, the machine learningmodule 812 acquires this data and based on the comparison with theprestored data of the one or more medical reference procedures, maytransmit a notification that the medical procedure is about to get over.The notification may be sent to one or more supporting staff team, otherpractitioners, and the like

In an embodiment, the machine learning module 812 is configured toidentify a next step for execution of the medical procedure based on thecomparison of the acquired data associated with usage of at least one ofthe item and the equipment during the medical procedure with theanalyzed prestored data associated with the respective at least one ofthe item and the equipment in one or more reference medical procedures.In some embodiments, the machine learning module 812 is configured toidentify the current step, as discussed above, to determine the nextsteps for execution of the medical procedure. For example, the machinelearning module 812 analyzes the prestored data and determines one ormore next steps in a medical procedure, for example, after the patientis anesthetized, glucose is to be given to the patient by using acatheter into the veins. The machine learning module determines, via theone or more mixed reality devices 132, whether the at least one of anitem and an equipment required during the execution of the identifiednext step of the medical procedure is present in a medical procedureroom. For example, at this stage in the above example, the machinelearning module 812 will determine whether the required catheter ispresent in the medical procedure room. Further, the machine learningmodule 812, transmit, via the procedure input module 804, a notificationto one or more devices 124, for example, one or more devices of medicalstaff, medical practitioner, inventory management team, other supportstaff, when the determined at least one of the item and the equipmentrequired during the execution of the identified next step of the medicalprocedure is missing from the medical procedure room. For instance, whenthe required catheter is determined to be missing, a notification willbe transmitted to the communication device 124.

In an embodiment, the machine learning module 812 is further configuredto compare the acquired data associated with at least one of theposition and the orientation of the at least one personnel for executionof the medical procedure with prestored data associated with therespective position and orientation of the at least one personnel in oneor more reference medical procedures, and provide recommendations foroptimal position or orientation of the at least one personnel forexecution of the medical procedure based on the comparison.

The continuous loop of the optimization functional block diagram 800, inpractice provides real time feedback and optimization of a procedure.For example, as the procedure moves along the procedure input module 804includes tracking of instruments that are highlighted, and arrows pointto the location in the room as needed through artificial prompt and dataanalysis. This continuous loop optimization provides for metric trackingto create safety, efficiencies, simplicity across all aspects of theprocedure. The metric will be able to produce specific and tailoredfeedback regarding the procedure as well as allow for real timeoversight (local or remote). Live feedback and machine learningsuggestions may be provided during this process. Messaging betweenmedical personnel 130 and medical practitioners may be automated andartificial intelligence triggered to notify the entire team of progressand current step of the procedure. For example, procedure tracking atthe procedure input module 804 may provide instructions such as bringingthe patient in. In another example, after the patient enters the medicalprocedure room, a message may be triggered that the patient will need tobe postponed for a period of time based on the machine learning dataexperience. Additional checks and balances can be set in placeincluding, for example, facial recognition, prompts to double check theincision site, and the like.

In practice, the procedure input module 804 may include communicationbetween the communication devices 124, the mixed reality devices 132,the optimization computing device 206, and/or the remote connections208. For example, the procedure input module 804 may indicate aninstrument missing from the medical procedure room and request a remoteconnection provide it. This provides for increased safety andsterilization procedures and increased efficiency, reduced turnovertimes for cases, communication and messaging across the team includingreal-time and automated, tracking/counting of consumables during aprocedure, and the like.

In practice, the procedure input module 804 includes acquiring of datavia one or more mixed reality devices 132. In operation, the one or moremixed reality devices 132 provide a view of the procedure room once theprocedures start so personnel will have an exact idea and view of whereeach member will be standing during the perioperative portion of theprocedure. Further, the one or more mixed reality devices 132 mayreceive information about medical procedure resources in visual mixedreality format, including quantities and tracking of these resources,use, patterns for use, sequence of use, percentage and priority of use.In an embodiment, the machine learning module 812 is configured tocompare the received input, via the module 802, for the medicalprocedure with the data acquired via the mixed reality devices 132,associated with the usage of at least one of an item and an equipmentduring the medical procedure. In an embodiment, the machine learningmodule 812 is configured to generate an alert when the data associatedwith the usage of at least one of an item and an equipment during themedical procedure is not consistent with the received medical procedure.In an exemplary embodiment, the machine learning module 812 isconfigured to generate warnings, prompts, alerts for counts, location,and management of missing instruments, devices, disposables on the fieldor within the patient as appropriate. For example, when the receivedinput indicates that the medical implants are to be handled with thehelp of a special tweezer, the machine learning module 812 is configuredto determine, based on the data acquired via the mixed reality devices132, whether the medical implants are handled with the help of thespecial tweezer in the medical procedure and generate an alert whenthere is a mismatch in the way of handling. In some embodiments, themachine learning module 812 is configured to identify, via the one ormore mixed reality devices 132, procedure instruments and devices from alibrary of existing and new instruments devices and disposables storedin the data module 810, track and store back in the data module 810those devices geographically on the medical field or within the patient,thereby enabling warnings, prompts, alerts for counts, location, andmanagement of missing instruments, devices, disposables on the field orwithin the patient as appropriate.

In an embodiment, the machine learning module 812 is configured todetermine data associated with an item or an equipment required in themedical procedure based on inputs received from the enablingtechnologies. In an embodiment, the input from the enabling technologiesis received via one or more input module. For example, the input modulemay be a remote input module 808 or a local input module 814. The inputfrom remotely present enabling technologies is received via the remoteinput module 808 and the input from locally present enablingtechnologies is received via the local input module 814. In an exemplaryembodiment, the data associated with the item includes at least anidentifier or a size of the respective item or equipment. For example,the machine learning module 812, may enable the medical procedure team,to measure and size medical instruments, implants and devices forpatient specific use given the data available by enabling technologiessuch as operating microscope, fluoroscope, navigational system, roboticsystem, endoscopic system, for that medical practitioner, for thatprocedure. Specifically, the one or more mixed reality devices 132 mayprovide for a data interface for managing gas, pharmaceutical andanesthetics quantity, dose, frequency, inventory, logistics andinventory ordering and replacement as linked to the externalhospital-based management system on behalf of the specific patient.

In operation, during a procedure, the one or more mixed reality devices132 may provide for an interactive data interface linking the medicalprocedure environment through the data exchange of a microscope,endoscopic system, fluoroscopic system, navigation/robotic system, forlive web-based conferencing, education and training, and surgicaldemonstrations for technology, remote surgery, or remote surgerydemonstration/education. In an embodiment, the input module is coupledto one or more medical enabling technologies and is configured tocontrol operations of the one or more medical enabling technologies. Inparticular, the machine learning module 812 is configured to control,via the input module, the operations of the one or more medical enablingtechnologies based on the stored data associated with the patientundergoing the medical procedure. For example, the machine learningmodule 812 is configured to control a specific anesthesia dosagerequired for a particular medical procedure based on the stored dataassociated with the patient. Specifically, the one or more mixed realitydevices 132 may provide for interactive data interface linking theanesthesia environment, gas exchange machines and monitors, anesthesiamachines and telemetry, gas rates and anesthesia measures, for purposeof tracking patient specific data related to adequacy, safety,prescription and optimization of anesthetic induction, maintenance, andrecovery.

Specifically, procedure inputs from the one or more mixed realitydevices 132 may provide direct external visualization of the operatingroom environment through the eyes of the medical personnel 130 wearingthe mixed reality device 132, through the external visualization of, forexample, a microscope, an endoscopic system, a fluoroscopic system, anavigation/robotic system to an external expert, product specialist, orother professional. In some embodiments, a three-dimensional aerialperspective view of the medical procedure room 102 suite allowspersonnel capability for predictive planning of layout of equipmentplacement, relative to patients position for surgical case that willsave on turnover times and efficiency for each surgical procedure andmedical practitioner preference.

Further, in some embodiments, the one or more mixed reality devices 132identifies whether one or more bed attachments are correct based on aright side or left side surgery (arm attachments and the like). Using avisual checklist, the one or more mixed reality devices 132 determinesand provides procedure inputs that additional items for the medicalpractitioner are correct and documented as well as location is saved.

In practice, the one or more mixed reality devices 132 may store andtransmit animation, videos, demonstrations and live coaching in a waywhere the specific medical practitioner, for the specific procedure, anda specific device or technology could receive education, consultation,and instruction of best use or trouble shooting remotely as indicated bya remote input module 808. This transmission may be stored information,live two-way transmitted information, could be interfaced with mixedreality ‘white boards’ and drawing/display tools and with virtualrepresentation of real instruments/devices either stored in the datamodule 810 or one or more remote connections (for example the remoteinput module 808). As described above, the input module may be theremote input module 808 and the local input module 814. The remote inputas illustrated by module 808, may be received from remote connections208 before, during, and after a particular medical procedure. Remoteinput of module 808, for example may include the integration ofmultimedia data management, acquisition and expression from secondaryenabling medical technologies. In an embodiment, the local input asillustrated by the local input module 814 may be received from localenabling technologies present in the medical procedure room.

Module 806 implements a natural language processing algorithm. Thenatural language processing algorithm collects unstructured data fromvarious sources including procedure input of module 804, remote inputsof module 808, and local inputs of the module 814. These inputs areconverted into machine-readable structured data, stored in the datamodule 810, and then analyzed by the machine learning module 812. Itwill be appreciated that the machine learning module 812, in someembodiments, may analyze the data in the data module 810 prior to a nextprocedure room set up 802, during a procedure itself to modify aprocedure room set up, and/or after a procedure for analytical andtraining purposes. The machine learning module 812 may analyze the datain the data module 810 at other instances now known or hereinafterdeveloped.

In practice, the machine learning module uses the data in the datamodule 810 for procedure efficiency, outcome and optimization. Themachine learning module 812 may, for example use medical practitionerspecific identification of the procedure room and the equipment andsupply characteristics for any given procedure, and any given patient,for optimization of procedure sequence, flow and execution, andultimately help to define lean simple system design for optimaloperating room setup and procedure steps for that specific medicalpractitioner and for that specific procedure. Practitioner specificidentification of the procedure room and the equipment and supplycharacteristics for any given procedure, and any given patient, wouldthen enable identification of experience based predictive data forperformance, safety, outcome efficiency and inventory control/managementfor medical practitioners learning new procedures/devices/systems onbehalf of their patients.

It will be appreciated that the optimization functional block diagram800 in implementation provides for medical practitioner specificidentification of the medical procedure room and the equipment andsupply characteristics for any given procedure, and any given patient,enabling machine learning processes for optimization of proceduresequence, flow and execution, and ultimately help to define lean simplesystem design for optimal medical procedure room setup for that specificmedical practitioner and for that specific procedure.

FIG. 9 is a flow diagram of a method of extended continuous optimizationusing the functionality of FIG. 8 in accordance with some embodiments.Specifically, FIG. 9 is a flow diagram 900 illustrating theimplementation of the optimization system 104 and various functionalityas described previously herein for one or more medical practitioners,one or more medical procedure rooms, and/or one or more medicalprocedures.

The flow diagram 900 begins with operation 902 wherein a procedureidentifier is initiated by setting the medical procedure identifier “P”equal to one “1”. Next, in operation 904, a medical procedure roomidentifier is initiated by setting the medical procedure room identifier“R” equal to one “1”. Next, in operation 906, a medical practitioneridentifier is initiated by setting the medical practitioner identifier“MP” equal to one “1”.

Next, in operation 908, optimization functionality occurs. It will beappreciated by those of ordinary skill in the art that the optimizationoperation 908 includes the optimization described previously herein forFIGS. 1 through 8.

After the optimization operation 908, the process of flow diagram 900continues to operation 910 in which it is determined whether there aremore medical practitioners. Specifically, it is determined whetherMP=MP+1 is to be included. When MP+1 is determined to be included, theprocess continues to operation 912 in which the MP identifier isincremented to MP=MP+1. The process then cycles back to the optimizationoperation 908. In operation, at this stage, the optimization system 104is configured to optimize the execution of one or more steps of themedical procedure based on the plurality of received medicalpractitioner identifiers (for example, MP and MP+1). For example, theoptimization system 104 is configured to obtain data associated withpreferred execution of the medical procedure associated with thereceived medical practitioner identifiers and optimize the execution ofone or more steps of the medical procedure based on the obtainedinformation. In some embodiments, the optimization may include mergingthe data associated with preferred execution of the medical procedureassociated with the received plurality of received medical practitioneridentifiers or selecting the data associated with preferred execution ofthe medical procedure associated with the primary/predefined medicalpractitioner, in case of any conflict. The optimized medical proceduremay then be associated with the received plurality of medicalpractitioner identifiers and stored in the data module 810.

When, in operation 910, it is determined that there is no MP+1 toinclude, the process continues to operation 914 in which it isdetermined whether there are more medical procedure rooms. Specifically,it is determined whether R=R+1 is to be included. When R+1 is determinedto be included, the process continues to operation 916 in which the Ridentifier is incremented to R=R+1. The process then cycles back to theoperation 906. In operation, at this stage, the optimization system 104is configured to optimize the execution of one or more steps of themedical procedure based on the plurality of received medical procedureroom identifiers (for example, R and R+1). For example, the optimizationsystem 104 is configured to obtain data associated with preferredexecution of the medical procedure associated with the received medicalprocedure room identifiers and optimize the execution of one or moresteps of the medical procedure based on the obtained information. Theoptimized medical procedure may then be associated with the receivedplurality of medical procedure room identifiers and stored in the datamodule 810.

When, in operation 914, it is determined that there is no R+1 toinclude, the process continues to operation 918 in which it isdetermined whether there are more medical procedures. Specifically, itis determined whether P=P+1 is to be included. When P+1 is determined tobe included, the process continues to operation 920 in which the Pidentifier is incremented to P=P+1. The process then cycles back to theoperation 904. In operation, at this stage, the optimization system 104is configured to optimize the execution of one or more steps of themedical procedure based on the plurality of received medical procedureidentifiers (for example, P and P+1). For example, the optimizationsystem 104 is configured to obtain data associated with preferredexecution of the medical procedure associated with the received medicalprocedure identifiers and optimize the execution of one or more steps ofthe medical procedure based on the obtained information. The optimizedmedical procedure may then be associated with the received plurality ofmedical procedure identifiers and stored in the data module 810.

When, in operation 918, it is determined that there is no P+1 toinclude, the process cycles back to operation 902 in which P is reset.In this manner, the optimization system and methods described hereinprovide continuous optimization for one or more medical practitioners,one or more medical procedure rooms, and one or more medical proceduresand any combination therein.

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings.

The benefits, advantages, solutions to problems, and any element(s) thatmay cause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeatures or elements of any or all the claims. The invention is definedsolely by the appended claims including any amendments made during thependency of this application and all equivalents of those claims asissued.

Moreover, in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has”,“having,” “includes”, “including,” “contains”, “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element preceded by“comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . .a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially”, “essentially”,“approximately”, “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (for example, comprising a processor) to performa method as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus, the following claimsare hereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

1. A system for optimizing execution of a medical procedure, the systemcomprising: a procedure module configured to: obtain at least onemedical procedure input associated with a medical practitioneridentifier, a medical procedure identifier, and a medical procedure roomidentifier for execution of the medical procedure, and provide virtualimage guidance, via one or more mixed reality devices, for execution ofthe medical procedure based on the received at least one medicalprocedure input; a procedure input module configured to: acquire, viathe one or more mixed reality devices, data associated with usage of atleast one of an item and an equipment during the medical procedure; anda machine learning module configured to: analyze prestored dataassociated with usage of at least one of the item and the equipment inone or more reference medical procedures corresponding to the one ormore of the medical practitioner identifier, the medical procedureidentifier, and the medical procedure room identifier, compare theacquired data associated with usage of at least one of the item and theequipment during the medical procedure with the analyzed prestored dataassociated with the respective at least one of the item and theequipment in one or more reference medical procedures, and providerecommendation for optimizing the execution of the medical procedurebased on the comparison.
 2. The system of claim 1, wherein the at leastone medical procedure input includes one or more of steps for executionof the medical procedure, order of steps for execution of the medicalprocedure, and instructions for one or more steps for execution of themedical procedure.
 3. The system of claim 1, wherein the data associatedwith usage of at least one of an item and an equipment includes: dataassociated with at least a pattern of use, a sequence of use, and apriority of use of at least one of the item and the equipment during themedical procedure, and further wherein the recommendation includes oneor more of changing the pattern of use, the sequence of use, and thepriority of use of at least one of the item and the equipment during themedical procedure.
 4. The system of claim 1, wherein the analyzingfurther comprises determining an optimal relationship of the dataassociated with usage of at least one of the item and the equipment inone or more reference medical procedures with the one or more of themedical practitioner identifier, the medical procedure identifier, andthe medical procedure room identifier, and further wherein the comparingcomprises determining whether the acquired data associated with usage ofat least one of the item and the equipment during the medical procedurecorrespond to the determined optimal relationship for the one or more ofthe medical practitioner identifier, the medical procedure identifier,and the medical procedure room identifier.
 5. The system of claim 4,wherein the machine learning module is configured to implement one ormore machine learning algorithms to determine the optimal relationshipof the data associated with usage of at least one of the item and theequipment in one or more reference medical procedures with the one ormore of the medical practitioner identifier, the medical procedureidentifier, and the medical procedure room identifier.
 6. The system ofclaim 1, wherein the machine learning module is configured to: identifya current step of the medical procedure that is being executed based onthe comparison of the acquired data associated with usage of at leastone of the item and the equipment during the medical procedure with theanalyzed prestored data associated with the respective at least one ofthe item and the equipment in one or more reference medical procedures,and transmit, via the procedure input module, a notification to at leastone device identifying the current step of the medical procedure.
 7. Thesystem of claim 1, wherein the machine learning module is furtherconfigured to: identify a next step for execution of the medicalprocedure based on the comparison of the acquired data associated withusage of at least one of the item and the equipment during the medicalprocedure with the analyzed prestored data associated with therespective at least one of the item and the equipment in one or morereference medical procedures; determine, via the one or more mixedreality devices, whether the at least one of an item and an equipmentrequired during the execution of the identified next step of the medicalprocedure is present in a medical procedure room; and transmit, via theprocedure input module, a notification to one or more devices when thedetermined at least one of the item and the equipment required duringthe execution of the identified next step of the medical procedure ismissing from the medical procedure room.
 8. The system of claim 1,wherein the procedure module is further configured to update the virtualimage guidance for the medical procedure based on the receivedrecommendation.
 9. The system of claim 1, wherein the machine learningmodule is configured to provide the at least one medical procedure inputcorresponding to one or more of the medical practitioner identifier, themedical procedure identifier, and the medical procedure room identifierfor execution of the medical procedure.
 10. The system of claim 1,further comprising: a data module configured to store data associatedwith a patient undergoing the medical procedure; an input module coupledto one or more medical enabling technologies and configured to controloperations of the one or more medical enabling technologies; and whereinthe machine learning module is configured to: control, via the inputmodule, the operations of the one or more medical enabling technologiesbased on the stored data associated with the patient undergoing themedical procedure.
 11. The system of claim 1, further comprising: a datamodule configured to store tutorials associated with execution of one ormore steps of the medical procedure, wherein the procedure module isfurther configured to provide virtual image guidance for execution ofthe medical procedure using the stored tutorials.
 12. The system ofclaim 1, wherein the procedure input module is further configured toacquire, via the one or more mixed reality devices, data associated withat least one of a position and an orientation of at least one personnelfor execution of the medical procedure, and further wherein the machinelearning module is configured to: compare the acquired data associatedwith at least one of the position and the orientation of the at leastone personnel for execution of the medical procedure with prestored dataassociated with the respective position and orientation of the at leastone personnel in one or more reference medical procedures, and providerecommendation for optimal position or orientation of the at least onepersonnel for execution of the medical procedure based on thecomparison.
 13. A method for optimizing execution of a medicalprocedure, the method comprising: obtaining, by a procedure module, atleast one medical procedure input associated with a medical practitioneridentifier, a medical procedure identifier, and a medical procedure roomidentifier for execution of the medical procedure; providing, by theprocedure module, virtual image guidance, via one or more mixed realitydevices, for execution of the medical procedure based on the received atleast one medical procedure input; acquiring, by a procedure inputmodule, via the one or more mixed reality devices, data associated withusage of at least one of an item and an equipment during the medicalprocedure; analyzing, by a machine learning module, prestored dataassociated with usage of at least one of the item and the equipment inone or more reference medical procedures corresponding to the one ormore of the medical practitioner identifier, the medical procedureidentifier, and the medical procedure room identifier; comparing, by themachine learning module, the acquired data associated with usage of atleast one of the item and the equipment during the medical procedurewith the analyzed prestored data associated with the respective at leastone of the item and the equipment in one or more reference medicalprocedures, and providing, by the machine learning module,recommendation for optimizing the execution of the medical procedurebased on the comparison.
 14. The method of claim 13, wherein the atleast one medical procedure input includes one or more of steps forexecution of the medical procedure, order of steps for execution of themedical procedure, and instructions for one or more steps for executionof the medical procedure.
 15. The method of claim 13, wherein the dataassociated with usage of at least one of an item and an equipmentincludes: data associated with at least a pattern of use, a sequence ofuse, and a priority of use of at least one of the item and the equipmentduring the medical procedure, and further wherein the recommendationincludes one or more of changing the pattern of use, the sequence ofuse, and the priority of use of at least one of the item and theequipment during the medical procedure.
 16. The method of claim 13,wherein the analyzing further comprises determining an optimalrelationship of the data associated with usage of at least one of theitem and the equipment in one or more reference medical procedures withthe one or more of the medical practitioner identifier, the medicalprocedure identifier, and the medical procedure room identifier, andfurther wherein the comparing comprises determining whether the acquireddata associated with usage of at least one of the item and the equipmentduring the medical procedure correspond to the determined optimalrelationship for the one or more of the medical practitioner identifier,the medical procedure identifier, and the medical procedure roomidentifier.
 17. The method of claim 16, further comprising: implementingone or more machine learning algorithms by the machine learning module,to determine the optimal relationship of the data associated with usageof at least one of the item and the equipment in one or more referencemedical procedures with the one or more of the medical practitioneridentifier, the medical procedure identifier, and the medical procedureroom identifier.
 18. The method of claim 13, further comprising:identifying, by the machine learning module, a current step of themedical procedure that is being executed based on the comparison of theacquired data associated with usage of at least one of the item and theequipment during the medical procedure with the analyzed prestored dataassociated with the respective at least one of the item and theequipment in one or more reference medical procedures, and transmitting,by the machine learning module, via the procedure input module, anotification to at least one device identifying the current step of themedical procedure.
 19. The method of claim 13, further comprising;identifying, by the machine learning module, a next step for executionof the medical procedure based on the comparison of the acquired dataassociated with usage of at least one of the item and the equipmentduring the medical procedure with the analyzed prestored data associatedwith the respective at least one of the item and the equipment in one ormore reference medical procedures; determining, by the machine learningmodule, via the one or more mixed reality devices, whether the at leastone of an item and an equipment required during the execution of theidentified next step of the medical procedure is present in a medicalprocedure room; and transmitting, by the machine learning module, viathe procedure input module, a notification to one or more devices whenthe determined at least one of the item and the equipment requiredduring the execution of the identified next step of the medicalprocedure is missing from the medical procedure room.
 20. The method ofclaim 13, further comprising: acquiring, via the one or more mixedreality devices, data associated with at least one of a position and anorientation of at least one personnel for execution of the medicalprocedure; comparing, by the machine learning module, the acquired dataassociated with at least one of the position and the orientation of theat least one personnel for execution of the medical procedure withprestored data associated with the respective position and orientationof the at least one personnel in one or more reference medicalprocedures, and providing, by the machine learning module,recommendation for optimal position or orientation of the at least onepersonnel for execution of the medical procedure based on thecomparison.